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A piloted simulation that studied the handling qualities for a precision lunar landing task 
from final approach to touchdown is presented. A vehicle model based on NASA’s Altair 
Lunar Lander was used to explore the design space around the nominal vehicle 
configuration to determine which combination of factors provides satisfactory pilot-vehicle 
performance and workload; details of the control and propulsion systems not available for 
that vehicle were derived from Apollo Lunar Module data. The experiment was conducted 
on a large motion base simulator. Eight Space Shuttle and Apollo pilot astronauts and three 
NASA test pilots served as evaluation pilots, providing Cooper-Harper ratings. Task Load 
Index ratings and qualitative comments. Each pilot flew seven combinations of control 
response types and three sets of displays, including two varieties of guidance and a non- 
guided approach. The response types included Rate Command with Attitude Hold, which 
was used in the original Apollo Moon landings, a Velocity Increment Command response 
type designed for up-and-away flight, three response types designed specifically for the 
vertical descent portion of the trajectory, and combinations of these. It was found that 
Velocity Increment Command significantly improved handling qualities when compared 
with the baseline Apollo design, receiving predominantly Level 1 ratings. This response type 
could be flown with or without explicit guidance cues, something that was very difficult with 
the baseline design, and resulted in approximately equivalent touchdown accuracies and 
propellant burn as the baseline response type. The response types designed to be used 
exclusively in the vertical descent portion of the trajectory did not improve handling 
qualities. 


Introduction 

H andling qualities are those characteristics of a flight vehicle that govern the ease and precision with which a 
pilot is able to perform a flying task. 1 The way in which particular vehicle factors affect handling qualities has 
been studied in aircraft for decades, 2 ’ 4 and reference standards for the handling qualities of both fixed-wing aircraft 5 
and rotary-wing aircraft 6 have been developed and are now in common use. These standards define a subset of the 
dynamics and control design space that provides good handling qualities for a given vehicle type and flying task. A 
new generation of spacecraft now under development by NASA to replace the Space Shuttle and return astronauts to 
the Moon will have a manual control capability for several mission tasks, and the ease and precision with which 
pilots can execute these tasks will have an important effect on performance, mission risk and training costs. No 
reference standards currently exist for handling qualities of piloted spacecraft. 

The simulation reported in this paper builds on the extensive work NASA’s Apollo Program conducted on 
manual control during lunar landing, which had principal goals of establishing simulation requirements for both 
research and training, selecting response types for attitude control, and designing the control system for 
compatibility with other manual control factors. 7 ' 15 A recent overview of the findings of these papers can be found in 
Ref. 16. The goal of the current study is to inform the design of any piloted vehicle designed to land on the Moon, 
and in particular NASA’s Altair Lunar Lander. A study conducted in May 2007 at the same facility used to generate 
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the results for this paper found that a precision lunar landing could not be executed with tolerable workload using a 
Rate Command with Attitude Hold (RCAH) response type and an Apollo Lunar Module (LM) model unless attitude 
guidance was present. 17 A subsequent simulation of a generic lander based on Altair, conducted in December 2008, 
studied much lower values of control power than were examined in the original Apollo studies as a function of 
rotation inceptor command sensitivity. 16 That study found improved handling qualities could be achieved by careful 
selection of the sensitivity for a given control power, but the handling qualities were still not satisfactory for routine 
use with the RCAH response type and low control powers. The results of the current study, which should not be 
interpreted as an evaluation of Altair’ s handling qualities, may be used to determine from a system perspective how 
to most easily achieve satisfactory handling qualities, whether by introducing new response types, changing displays 
or increasing control power. The specific contributions of this paper lie in the evaluation of novel response types and 
display elements, including several guidance schemes, at a control power that is 15% of that used by the Apollo LM. 
The task is to conduct a precision landing with and without guidance, which may be required during future lunar 
outpost missions. Precision landing was not a requirement for the LM. A principal goal of the current simulation is 
to examine whether handling qualities are satisfactory with improved response types and displays without resorting 
to increasing the size of the reaction control system (RCS) jets to increase control power. In this experiment, the 
vehicle design is updated with the mass properties, geometry, visual field of view and propulsion system (both RCS 
and descent engine) specifications to match that of the Altair design in August 2009; the displays, inceptors, visual 
cues, control response types and guidance algorithms were all designed from scratch because they were not available 
for Altair. The goals of the study are to determine which control response types and display features improve 
handling qualities the most, which of these configurations allows the landing task to be completed without guidance, 
and how the configurations affect touchdown performance and propellant use. 

The paper begins with a discussion of the experiment design, which includes details on the flying task, test 
matrix and procedure. A description of the vehicle design follows, where the lunar lander dynamics and control 
model is detailed along with four new control response designs. The motion-base simulator is described next, along 
with the cockpit instrumentation, guidance and displays, and piloting procedures. Results are then presented, and the 
paper concludes with a summary of key findings. 

Experiment Design 

A piloted simulation in November 2009 assessed the handling qualities for a precision lunar landing task from 
terminal descent to touchdown. The primary experiment variables were the control response type and display 
feature set. These variables are expected to be the most crucial for determining handling qualities if the control 
power of the vehicle is held constant. While the control power must be selected early in the design process and is 
difficult to change because of mass implications, the response type and displays may be modified in software to 
compensate for deficiencies known to result from low thrust levels. 

Flying Task 

The Altair vehicle is designed to carry four astronauts and cargo from lunar orbit to the surface and return only 
the people to orbit in an ascent stage that is separable from the lower, descent portion of the lander. This experiment 
evaluated handling qualities for a precision landing task, from final approach through terminal descent to 
touchdown, following a trajectory based on the nominal Apollo trajectory. Changes to the vehicle trajectory were 
made by firing opposing RCS jets to alter the attitude of the lander and correspondingly tilt the descent engine’s 
thrust vector. Feedback guidance laws developed in a previous simulation for the approach and landing task were 
used again here, 17 and the corresponding velocity or attitude guidance cues were displayed to the pilot via cockpit 
instrumentation. 

The task began at 500 ft altitude with a forward speed of 60 ft/s and a descent rate of 16 ft/s; for Apollo missions 
this was known as “low gate” and represented the point on the trajectory where the manual flying phase would 
begin. 18 At this point, the spacecraft was at 1,350 ft range from the designated touchdown point, and pitched up 16 
deg. The desired trajectory brought the spacecraft to a level attitude directly above the touchdown point, at an 
altitude of 150 ft with a descent rate of 3 ft/s. This rate of descent was held constant until one of the 6-ft probes 
attached to the lander legs made contact with the lunar surface. A shut-off command was then sent to the main 
engine, and the vehicle dropped until the legs settled on the lunar surface. This reference trajectory profile is 
illustrated in Fig. 1. For comparison, it also shows the uncontrolled trajectory that would result if no pilot inputs 
were made starting from an initial condition with vertical force equilibrium. 
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Figure 1. Reference trajectory profile in the vertical plane 


The dynamics of the trajectory described above are confined to the vertical plane, with pitch attitude as the 
primary means of longitudinal trajectory control. In order to excite lateral dynamics, the initial condition included a 
lateral offset of 250 ft from the touchdown point so that the initial velocity vector did not point directly at the 
landing site. This lateral offset required the pilot to use roll attitude as a means of lateral trajectory control. The 
pilot similarly controlled pitch attitude to regulate the longitudinal trajectory, but the vertical dimension was 
nominally controlled by the automatic system and the option of taking manual vertical control left to the pilot’s 
discretion. 


Experiment Matrix 

The experiment matrix was designed to answer the question of whether satisfactory handling qualities could be 
achieved with more highly-augmented control response types, better heads-down displays, or combinations of both. 
Two classes of control response types were developed, one class appropriate for the entire trajectory from low gate 
to touchdown but optimized for the up-and-away portion, and another class optimized for the final vertical descent 
portion of the trajectory from 150 ft altitude to touchdown. This addresses the question of whether forcing the pilot 
to change his method of control halfway through the task creates more difficulty than the presumed benefit of 
having response types that are better tailored to the particular characteristics of the two portions of the trajectory. 

The displays used in this simulation were adapted from aircraft glass cockpit and advanced rotorcraft designs; 19 
the latter was used because of the perceived similarities between lunar landing and rotorcraft landings. These 
displays attempt to present vehicle state information in an intuitive way, and significant effort was devoted to 
scaling the display symbols’ relative positioning to allow the pilot to fly the task without explicit guidance. 
Configurations were tested both with guidance, tailored to the particular up-and-away response type, and without 
guidance to ascertain the need for this extra feature as a function of control response type. 

The order in which the pilots flew each configuration was carefully selected to minimize training effects. Each 
pilot flew the guidance cases before flying any non-guidance cases, and evaluated all of the configurations of one of 
the up-and-away response types before evaluating the configurations of the other up-and-away response type. The 
response type presented first was alternated for each pilot. Evaluations of the configurations for a given up-and- 
away response type were presented in random order to each pilot. Once all the guidance cases had been evaluated 
the pilots flew the three Velocity Increment (VINC) configurations without guidance, and with the time remaining 
flew as many RCAH configurations without guidance as possible. All 14 configurations of the experiment are 
shown in Table 1, however not every pilot was able to complete all the non-guidance configurations, so those are 
considered part of the secondary matrix. All pilots completed each of the guidance configurations. 
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Table 1. Experiment matrix, with configurations not evaluated in black 
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Evaluation Pilots and Training 

Eleven highly trained test pilots, comprising six Shuttle pilot astronauts, two Apollo LM pilots and three NASA 
test pilots served as evaluation pilots. All pilots were male and had substantial training and experience as test pilots, 
logging an average of 7,200 hours on various fixed/rotary wing and powered-lift aircraft. Relevant experience also 
included thirteen Shuttle missions as pilot, ten as Shuttle commander and two Apollo missions. 

Pilots were allowed a two-hour training and familiarization session in the simulator cockpit in which they 
practiced the flying task for each of the configurations with guidance and the Velocity Increment mode without 
guidance. Training for the non-guided RCAH response type was left until after data collection in the primary matrix 
was completed. The pilot was allowed to continue training until he judged his performance was consistent and he 
had developed appropriate piloting strategies. 


Table 2. Upper limits of desired / adequate touchdown performance 


Touchdown Parameter 

Desired 

Adequate 

Range to Target Point 

15 ft 

25 ft 

Horizontal Velocity 

2 ft/s 

4 ft/s 

Descent Rate 

6.5 ft/s 

8 ft/s 

DE Propellant Burn 

<65% 
(1760 Ibm) 

<90% 
(2440 Ibm) 

RCS Propellant Burn 

290 Ibm 

400 Ibm 

Roll Angle 

3 deg 

6 deg 

Pitch Angle 

3 deg 

6 deg 

Roll Rate 

3 deg/s 

6 deg/s 

Pitch Rate 

3 deg/s 

6 deg/s 

Yaw Rate 

1 deg/s 

1 .5 deg/s 
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Data Collection Procedures 

The pilots were required to fly at least two data runs in each configuration, with the option for as many 
additional runs as desired, and then provide Cooper-Harper handling qualities ratings, 1 NASA Task Load Index 
ratings 20 and qualitative comments. The ratings were based on both the quantitative touchdown performance (see 
Table 2) and a qualitative evaluation of the manner in which the vehicle got to the end point. This overall 
assessment of desired, adequate, or inadequate performance was judged for the complete task of approach and 
landing. The values of desired and adequate performance bounds for key parameters were obtained from a survey of 
Apollo Lunar Module literature, the primary consideration being confidence that adequate performance would not 
result in the vehicle tipping over or damaging the legs; the 15 ft range error limit for desired performance is roughly 
comparable to the required landing accuracy for the automated system during normal conditions (10 ft). 

A debrief session was held after all test configurations had been evaluated. The pilots were asked to fill out a 
one-page questionnaire designed to elicit high-level comments on cockpit displays, out-the-window displays, 
guidance cues, control response, and experiment design. The pilot’s participation in the study concluded with a 
discussion of key insights and suggestions for future work. 

Lunar Lander Dynamics and Control Model 

NASA’s Altair vehicle 21 was in the early design stages when model development was finalized in October 2009, 
so the model is based only on available propulsion system characteristics, mass properties and geometry of the new 
vehicle. Some of the remaining aspects of the model, including the RCAH control system design and switching 
curve logic, approach trajectory, and touchdown performance requirements, were based on Apollo LM data from 

14 22 23 

several sources, ' ’ and the remaining aspects, including new control response types, displays, inceptors, and RCS 
prioritization logic were designed from scratch. Shown in Fig. 2 is the coordinate frame and schematic of the Altair 
vehicle. 


Vehicle Mass/Inertia Model 

The mass of the vehicle when the task begins is 55,900 lbm; it then varies due to consumption of propellant by 
the descent engine and RCS jets. During the final approach to touchdown phase the vehicle mass decreases by only 
5% due to propellant consumption. Hence in this model it is assumed that moments of inertia are constant and that 
the vehicle center of mass (c.m.) location remains constant. Propellant slosh effects at the low angular accelerations 
available in this model were found to be negligible in a previous study 16 and were not modeled here. 
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Figure 2. Schematic of Altair Lander 
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Descent Engine 

The descent engine is the spacecraft’s main rocket engine, with a specific impulse of 449 s. For the landing task, 
its thrust force is used to regulate the descent rate and to apply coarse trajectory control in the horizontal plane by 
rolling and/or pitching the vehicle. In this model, the engine does not gimbal, and the thrust line passes through the 
vehicle c.m. Propellant mass budgeted for the piloted segment of the landing trajectory, including reserves, is 4774 
lbm. 

The descent engine thrust is directed along the negative body z-axis. During the flight phases from approach to 
touchdown, this thrust can be controlled by a throttle between 10% and 100% of the maximum value of 18,627 lbf. 
The thrust command, T cmd , consists of two parts: T" m] and A T cmd . T° md is automatically computed as the force 
whose vertical component balances the vehicle’s lunar weight, mg lunar , while compensating for vehicle roll ( 0) and 
pitch (9) angles: 


nip, 



cos $ cos 9 


( 1 ) 


The secondary part of the thrust command, A T cmd , is an increment derived from pilot input to the vehicle’s rate of 
descent (ROD) command. In this command mode each inceptor “click” (movement of the inceptor out of detent and 
back into detent) increments the commanded descent rate by +1 ft/s; the descent rate is regulated within a deadband 
of +0.1 ft/s by a proportional feedback controller with a time constant r= 1.5 s. 


ATT 


r K md ~h] 


cos<f>cos9 


( 2 ) 


Engine response to thrust commands is modeled as a first-order system, with a time constant of 0.7 s. Hence the 
actual thrust produced by the descent engine, T DE , lags the commanded thrust T cmd = T° mi + A T cmd . 


Reaction Control System Jets 

There are four RCS pods on Altair’s descent module, as shown in Fig. 2. The RCS jets cannot be throttled; their 
response to a command input begins with a pure delay of 10 ms followed by a first order thrust response that reaches 
full thrust within an additional 13 ms. The RCS jet must then fire for a minimum of 40 ms, after which the thrust 
decays as a first-order response with time constant 110 ms. Propellant mass budgeted for the piloted segment of the 
landing trajectory, including reserves, is 444 lbm. RCS jets in this experiment are used exclusively for three-axis 
attitude control; the jets are never used for direct translation. 

Inceptor Shaping 

The state values commanded by the pilot were calculated by shaping the raw displacement of either the 
rotational hand controller (RHC) or translational hand controller (THC) using a linear-quadratic function, the same 
shaping function used in Apollo. This function is simply a weighted average of a linear function and a quadratic 
function: 


= oj, 


r (S-Sdb) + c lq (8-Sdb) 2 ^ 
(l-S DB ) + c LQ a-S DB ) 2 


if S> S n 


else =° 


(3) 


where c LQ is the linear-quadratic shaping parameter, S DB is the inceptor detent deadband, and o) cmd denotes the 
state value commanded at full throw of the inceptor ( 8 = 1 ). The parameter values were c LQ =1, the RHC deadband 
was 10% of full throw, the THC deadband was 30% of full throw, and oj (rnd varied depending on the state variable 
being commanded by the particular inceptor. 
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Attitude control 

Pilot attitude inputs are made with a three-axis RHC. This control inceptor is used for attitude stabilization and 
control along all three body axes; however, mode control is determined simultaneously for the roll and pitch axes 
(they are always in the same mode) and independently for the yaw axis. Whereas roll and pitch commands may be 
generated from either of the below response types, the yaw response type is always RCAH. 

Rate Command with Attitude Hold 

This mode is in effect when the rate command in a particular axis is changing very quickly (in “response to 
urgent commands,” 14 which for Apollo was a change in rate command of 6 deg/s in one second) and the attitude rate 
error is outside a 0.3 deg/s deadband. Error signals in rate mode are generated as the difference between the actual 
and desired angular rates: 


' P err 1 


P Pcmd 

Verr 
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C 1 ~ tfcmd 

. T err \ 


T — Y 

cmd 


where p, q, r, are the roll, pitch, and yaw rates respectively along the vehicle body axes. The commanded rates are 
computed from the inceptor deflection and the inceptor shaping function described in the previous section. 

When the attitude rate error drops below 0.3 deg/s and the RHC is not changing positions rapidly the control 
mode changes to attitude hold. The error signals for attitude hold mode are given by: 
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where (/), 6, and (//are the current values of the vehicle Euler angles, and <j ) cmd , 0 cmd and yf cmi are equal to the sum of 
the Euler angle values captured when attitude hold mode was last entered (for the particular axis) plus the integral of 
the commanded rates. That is. 


0cmi =</>(.to ,P )+ f p^dt 

Jt 0 

9 a mi = ) + f R , r Vend dt (6) 

Jt Q 

Yand =Y(to)+ f Y r cmd dt 

J 'o 


where <p{t plp ) and 0(t plp ) are the Euler roll and pitch angles captured when those axes last entered attitude hold 
mode, ) is the yaw angle captured when that axis last entered attitude hold mode, and p cmd , q cmd and r cmd are 
the time-dependent attitude-rate commands from the RHC. Control moment commands are generated about the 
appropriate axes in accordance with the phase-plane relationship between error signals, as illustrated in Fig. 3 for the 
pitch axis. 14 In that figure, a P is the nominal angular acceleration achieved with two RCS jets (approximately 0.65 
deg/s 2 ), 0 DB = 0.3 deg is the deadband for pitch attitude error, o = 0.8 deg is the magnitude of the flat boundary 
that enlarges the deadband region, q HI =1.4 deg/s is the rate error at which the control system transitions from two- 
jet to four-jet control, and k = 0.01 denotes the trade-off between RCS jet propellant consumption and error settling 
time. The phase-plane relationships for the roll and yaw axes are identical except for a small difference in the 
angular acceleration, a. 
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Attitude Error (deg) 

Figure 3. Switching curves for Rate Command with Attitude Hold 


Acceleration Command 

Acceleration command (ACEL) is related to attitude command through the inverse sine function, and so is 
essentially a rotational response type despite its label. For this reason the acceleration command response type was 
implemented on the RHC. It was only available to the pilot during the vertical descent portion of the trajectory (for 
given experiment configurations) because only that portion requires a nominally zero value of translational 
acceleration; the up-and-away trajectory requires a time-varying, non-zero value of acceleration so the pilot task in 
those circumstances would be to hold the inceptor out of detent against a spring force, a situation that was 
unacceptable to the development pilots. During the vertical descent portion of the trajectory the pilot can effect 
small changes in velocity with pulse inputs to the RHC, a technique that was much more acceptable than a 
continuous displacement. 

Acceleration command is implemented as a simplified version of the RCAH response type: only a single mode 
is available (attitude hold) and the raw value being output by the RHC is interpreted as a linear-quadratically shaped 
acceleration command, which is translated into roll and pitch commands via the following set of equations. 
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Toe cost/) 


1 cmd 
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where m is the current mass of the vehicle, Tde is the current thrust of the descent engine, and the longitudinal and 
lateral acceleration commands are u cmd and v cmd , respectively. The maximum available acceleration was 2 ft/s 2 , 
which corresponds to about 20 degrees of roll or pitch, and the commanded attitudes calculated from Eq. 7 were 
used with zero commanded attitude rates in the switching curve logic of Fig. 3. The difference between the RCAH 
and acceleration command switching curves are, in the case of the latter, use of four jets whenever possible (not 
switching to two jets when the attitude rate error was small), no flat portion of the inner switching curves, and a 
time-propellant tradeoff value of k= 0.25. 
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Translation Control 


Control of the vehicle’s translation rate was always done in the simulation through attitude commands rather 
than direct translation. The RCS jet thrust tested in this simulation was so low that firing the jets in parallel to 
achieve a pure translation force had little effect on translational velocity within the task time constraints. The 
following control response types commanded translational quantities (position and velocity) through manipulation 
of the pilot’s THC. 

Velocit\' Increment Command 

The Velocity Increment Command (VINC) response type was available during both the up-and-away and 
vertical descent portions of the trajectory for the appropriate experiment configurations. Conceptually very simple, 
the VINC mode allowed the pilot to command a discrete change in the vehicle's horizontal velocity with each 
discrete movement of the inceptor out of and back into detent. These discrete movements are referred to as “clicks” 
of the inceptor. Each click commanded a change in velocity in one axis of 10% of the total horizontal velocity; this 
scaling provided a straightforward way to increase the fineness with which the pilot could change velocity as they 
got closer to the touchdown target and as their velocity was reduced. A minimum velocity increment of 1 ft/s was 
established as a lower limit for each “click.” A blue ‘X’ was displayed on the pilot’s horizontal situation display 
indicating the current value of the velocity command. 

VINC was implemented on the THC and the velocity changes were aligned with the inceptor axes. The roll and 
pitch commands to the control system were calculated using Eq. 7, and the acceleration commands were related to 
the velocity errors in each axis through the following transfer function: 


a cmd _ S ^ 

v err j + ISO 


( 8 ) 


The attitude errors were then calculated from Eq. 5, the attitude rate errors were simply the attitude rates themselves 
because the target rate was zero, and the RCS firings were controlled by the switching curve logic expressed in Fig. 

3 subject to the same caveats given for the acceleration command system. The maximum roll or pitch angle that the 
control system would command, regardless of the velocity error, was 30 degrees per axis. 

Translation Rate Command With Position Hold 

The Translation Rate Command with Position Hold (TRCPH) response type is a highly augmented control 
mode that was adapted from advanced rotorcraft control systems. It is the translational analog to the RCAH 
response type for attitude: when the THC is out of detent the pilot is commanding a translational velocity 
proportional to inceptor displacement, and when it is back in detent the control system automatically holds a lateral 
position. The maximum velocity that could be commanded was 5 ft/s at full throw of the THC, and the raw inceptor 
output was shaped with a linear-quadratic filter. This response type was only available for the vertical descent 
portion of the trajectory in certain experiment configurations because of its limited velocity range. The control 
mode would change from translation rate command to position hold whenever the THC was in detent (commanding 
zero rate) and the total horizontal velocity was under 1.5 ft/s. No explicit indication of the commanded translational 
velocity was shown to the pilots on their heads-down displays. 

The implementation of TRCPH is identical to that of VINC mode with the exception of the shaping filter. 
Whereas the VINC mode commanded a cumulative velocity based on all previous inputs, TRCPH commanded a 
single velocity based only on the current displacement of the THC. The transfer function used to calculate an 
acceleration from a given velocity error is shown in Eq. 8, the attitudes required for given acceleration commands 
are given by Eq. 7, and the same saturation limits of 30 degrees per axis apply. For the position hold feature of this 
response type a separate, second order transfer function is used to calculate the acceleration command for a 
particular position error; otherwise the rest of the logic remains the same. The transfer function from position error 
to acceleration command in each axis is 


®cmd ^ 0-2 

= b ~7 

X err .V + 4 S + 30 


( 9 ) 
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where x err is the position error in a particular horizontal direction relative to the vehicle’s yaw angle. Both sub- 
modes of TRCPH used Eqs. 5 and 7 to calculate attitude errors for the switching curves with zero commanded 
attitude rate. 

Incremental Position Command 

The Increment Position Command (IPC) response type was the most highly-automated control system tested in 
this experiment, and it can be likened to a “target designator” system in which the pilot simply designates a landing 
point and the vehicle determines how to roll and pitch to get to that point. This mode is only available in certain 
configurations for the vertical descent portion of the trajectory, and when the pilot engages the response type the 
control system first enters a “velocity-nulling” mode. This intermediate mode, during which all pilot inputs are 
inhibited except for commands to change response types, is active as long as the translational velocity is above 1.5 
ft/s. At this time the system is essentially in a translation rate command response type with a velocity command of 
zero. When the velocity drops below 1.5 ft/s and the control system begins to actively hold position a blue box 
appears on the pilot’s horizontal situation display indicating the currently-held position. Each time the pilot moves 
the THC out of detent and back into detent (one “click” as discussed in the VINC section) the held position is 
incremented 5 ft in the appropriate direction. In an early implementation of this response type the blue box 
indicating held position was not available, so the first four pilots evaluated this response type without the aid of the 
touchdown point designator. 

This response type is implemented using the transfer function of Eq. 9 to calculate an acceleration command, 
saturated at +2 ft/s 2 , for a given position error in a particular axis. That acceleration command is converted to an 
attitude command using Eq. 7, which is then converted to RCS firing sequences using the switching curve logic 
discussed for the acceleration command response type with zero commanded attitude rate. 

Guidance Laws 

The velocity and attitude guidance laws used in this experiment were adopted without modification from an 
earlier lunar lander simulation: the detailed derivation may be found in Ref 17. These laws were designed to guide 
the pilot along a reference trajectory (see Fig. 1) from final approach through terminal descent to lunar touchdown. 

Attitude guidance cues are presented to the pilot as errors from the desired vehicle states. These errors are 
computed as the differences between the guidance and actual values of the roll and pitch angles. The velocity 
guidance cues are presented both as errors (on the pilot’s vertical situation display) and as absolute quantities (on the 
horizontal situation display). 


Simulation Environment 

The experiment was conducted in the Vertical Motion Simulator (VMS) at NASA Ames Research Center. The 
VMS is a large motion-base simulator 24 that has been used for numerous handling qualities evaluations. 25 Six 
degree-of-freedom simulator motion was used for the experiment because the reference trajectory was dynamic, 
featuring translational accelerations up to 0.35 ft/sec 2 and roll/pitch angular accelerations up to 1.3 deg/sec 2 ; the 
VMS cab also provides excellent visual and auditory cues. 

The simulator provided a similar cockpit configuration to that used in Apollo, as illustrated in Fig. 4. The 
evaluation pilot occupied the left station; the right station was occupied by the experimenter during training runs but 
not data collection runs. Each pilot station provided a three-axis RHC and a three-axis THC mounted on the right 
and left armrest, respectively. A button on top of the RHC toggled between manual and automatic control of the 
vehicle’s rate of descent. Up/down motion of the THC adjusted the commanded value of the rate of descent as long 
as manual vertical mode had been engaged. 

A simulated view of the lunar landscape was projected on a set of five non-collimating flat screen rear projection 
color displays. The display had a large field of view: 77 deg vertical and 225 deg horizontal. Masking was applied in 
front of the out-the-window displays to restrict the usable field of view to be consistent with Altair. This restriction 
is a departure from previous lunar lander experiments in the VMS, however, the precision landing task was 
essentially head-down, and the pilot’s attention was focused primarily on the cockpit instrumentation rather than the 
view outside the cockpit. 
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Downward-facing 
camera view 


Figure 4. Simulator cockpit layout 


Cockpit Instrumentation 

Cockpit displays were mounted on a console providing two nine-inch flat-panel monitors at each pilot station 
and a 15-inch flat-panel monitor in the center. The pilot station displays are shown in Fig. 5; the center monitor 
showed a camera view positioned five feet in front of the descent engine nozzle and oriented along the body +z 
direction (downward-facing). 

The screen on the pilot’s left is referred to as a Horizontal Situation Display (HSD); many aspects of this display 
were adapted from advanced rotorcraft displays that have been shown to improve handling qualities for landings 
without a clear view of the desired touchdown point. 19 The white dot in the center of the HSD represents the 
spacecraft (ownship), and the green pentagon in the shape of a “home plate” or “dog house” represents the landing 
site. The green line pointing forward from the ownship represents the vehicle’s horizontal velocity vector, and the 
open green circle is the vehicle’s current horizontal acceleration. That acceleration cue is referenced to the tip of the 
green velocity vector so the line always grows or shrinks in the direction of the green circle. The current yaw angle 
is represented by a white arc near the top of the HSD; that arc also provides a scale for the range and velocity 
indicators (currently 1500 ft and 120 ft/s in Fig. 5). To avoid symbol clutter when close to touchdown, the HSD 
undergoes two discrete scaling steps at velocities of 30 ft/s and 10 ft/s. The current range to target is next to a green 
“R” on the left side of the HSD, while altitude information is immediately to the right. The large green “421” in Fig. 
5 represents the current altitude in ft. The green bar just to the left of the altitude readout represents the altitude rate, 
with each of the white tick marks representing five ft/s. A digital readout of the altitude rate is shown below the 
green bar. The white diamond superimposed on top of the green altitude rate bar represents the altitude acceleration 
and is referenced to the tip of the rate bar; in this example the diamond is above the tip of the tape, which shows that 
the tape will grow towards the diamond and therefore the descent rate will be reduced. The throttle setting of the 
descent engine is represented by a blue tape on the far right of the screen, and the propellant remaining (referenced 
to 100% at the start of the run) is below the throttle in green. Annunciators at the top of the HSD tell the pilot what 
the up-and-away response type is (left), what the vertical descent response type is (center), and whether the vertical 
trajectory is being manually or automatically controlled (right). At the bottom of the screen are a yaw rate indicator 
and a touchdown indicator light. When velocity guidance is present a magenta semicircle on the HSD represents the 
guidance velocity, so the pilot’s job is to place the tip of the velocity vector in the center of the magenta “cup.” The 
blue ‘x’ represents the commanded velocity when in Velocity Increment mode. 

The right-side screen is called the Vertical Situation Display (VSD) and was adapted from standard glass cockpit 
designs of modern aircraft; it includes a pitch ladder, roll and yaw indicators and digital readouts of the same in the 
upper left of the display. The small triangles on the tick marks above and to the right of the pitch ladder indicate the 
body roll and pitch rates. Each tick mark on the scale is five deg/s. On the right side of the VSD is a set of tapes 
indicating altitude information; its function is identical to that available on the right side of the HSD. The tape on the 
left side of the display indicates the total horizontal velocity, in this case it reads between 54 and 55 ft/s. The 
annunciators at the top of the VSD are also identical to those on the HSD. The green circle with short line segments 
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representing “wings” and a “tail” is the flight path marker (FPM), which indicates the current direction of the total 
velocity vector; the black triangle to the right of the FPM is a conformal representation of the touchdown location: 
the desired touchdown point is at the point of the triangle and the top is at 150 ft of altitude. Attitude guidance 
needles, present only with the RCAH response type, are shown in magenta and are referenced to the black dot at the 
center of the VSD. In Fig. 5 the guidance needles indicate the pilot needs to pitch down several degrees and roll 
right approximately seven degrees. 



Piloting Technique 

The following procedures were reviewed with each pilot during the initial briefing. The pilots practiced these 
procedures during the training and familiarization simulator session prior to data collection. 

Vertical Speed Control 

Vertical trajectory control is accomplished by manual or automatic commands to the descent engine throttle. The 
simulation task began in automatic vertical control mode, which reduced the descent rate from an initial value of 16 
ft/s at 500 ft altitude to 3 ft/s at approximately 150 ft altitude in the vicinity of the landing site and maintained that 
all the way to touchdown. Pilots were allowed to switch to manual mode (by pressing a button on top of the RF1C) if 
they needed extra time to establish the vehicle in a stable vertical descent above the landing site. 

Horizontal Speed Control 

Florizontal control of the lander was always done using attitude commands, whether explicitly using the RHC or 
implicitly through the control system using the THC. The pilots switched to the vertical descent response type when 
they were within 50 ft horizontally of the touchdown point and below a translational velocity of 5 ft/s; this response 
type change was accomplished by sliding a switch on top of the RF1C. During data collection pilots were free to 
make their own decision as to when to make the control mode change, but they were required to make a switchover 
at some point during any run that included two response types. 

The piloting task was always heads-down using the VSD and F1SD, and the precise piloting requirements 
depended on the configuration of response types and guidance. When attitude guidance was present the pilot was 
required to center flight director needles on the VSD using the RHC, frequently using a pulse technique to make 
precise inputs. When velocity guidance was present the pilot would use the THC to command a particular velocity 
(represented on the HSD) to lead the velocity vector and maintain it within the magenta velocity guidance symbol 
(“cup”). When no guidance was present in the velocity command modes the pilot would try to slow down according 
to a mental schedule of ranges and velocities, frequently using the relative positions of the touchdown symbol and 
velocity vector as a guide. 
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Results 

Data collection was conducted with eleven pilots between October 26 and November 13, 2009. The subjective 
handling qualities ratings given by the pilots included Cooper-Harper ratings and NASA TLX ratings. Objective 
data were captured during the simulation on the touchdown range dispersions and RCS propellant usage. Because of 
the large number of experiment configurations tested only select results will be presented. The response type 
abbreviations used in this section are Rate Command with Attitude Hold (RCAH), Velocity Increment Command 
(VINC), Acceleration Command (ACEL), Translation Rate Command with Position Hold (TRCPH), Incremental 
Position Command without predicted touchdown point (IPC), Incremental Position Command with the predicted 
touchdown point (IPC-TD), attitude guidance (Attitude), velocity guidance (Velocity) and no guidance (State). 


Cooper-Harper Handling Qualities Ratings 

Pilots use the Cooper-Harper scale to assign handling qualities ratings from 1 (best) to 10 (worst) based on their 
assessment of task performance and effort. It is an ordinal scale, which means, for example, that the difference 
between ratings of 1 and 2 is not the same as the difference between ratings of 3 and 4. Ratings of 1. 2, and 3 on the 
Cooper-Harper scale correspond to Level 1 handling qualities, which are a general requirement for normal 
operations of flight vehicles. Ratings of 4, 5, and 6 correspond to Level 2, which may be acceptable for some off- 
nominal conditions, and ratings of 7, 8, and 9 correspond to Level 3, which is acceptable only for transition to a safe 
mode after a major failure/disturbance. Desired performance is necessary (but not sufficient) for Level 1 ratings, 
and adequate performance is necessary (but not sufficient) for Level 2 ratings. A rating of 10 indicates that control 
would be lost in that configuration. 

The CHR data for all the cases in which guidance was available is presented in Figure 6, and groups in that 
figure correspond to the same vertical descent response types. The colors — green, yellow and red — represent 
handling qualities Levels 1. 2 and 3. The shades of those colors represent different ratings within a given Level, 
where lighter colors are better ratings; black represents a rating of 10. The x-axis labels in that figure refer to the up- 
and-away response type, the vertical descent response type, and the type of guidance available to the pilot. 



■ CHR 10 

■ CHR 9 

■ CHR 8 

■ CHR 7 
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Figure 6. Cooper-Harper ratings for all configurations with guidance present 


Comparing the first two bars of Fig. 6 shows that the configuration in which the pilot flew the VINC response 
type all the way to landing using velocity guidance has substantially improved handling qualities relative to the 
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baseline RCAH response type flown with attitude guidance. Two-thirds of the pilots gave the pure VINC mode a 
Level 1 rating, while only three out of eleven rated RCAH Level 1 . The median ratings for those two configurations 
are relatively close but still in different Levels, with CHRs of 3 and 4, respectively. Interestingly, the worst rating 
for the pure VINC mode was a CHR 6, while the worst rating for pure RCAH was a 4. Two of the pilots who gave 
the pure VINC mode a Level 2 rating objected to the lack of direct control over the lander’s attitude; they felt that 
commanding a higher-order state like velocity and allowing the automatic system to control attitude was giving up 
too much control over the flight path and trajectory of the vehicle even though they found direct control of attitude 
to be a more technically demanding task than control of velocity. Research with pilots more familiar with each 
response type is needed to determine whether this objection would grow or disappear with additional training. 

Another important comparison that may be made in Fig. 6 is the relative utility of a second response type 
designed for the vertical descent portion of the trajectory: additional mental demand is associated with changing 
piloting technique halfway through the task. With the exception of the configuration RCAH/IPC/Attitude, handling 
qualities degrade every time the pilot is required to change response types for the second half of the task. Pilots 
explained this degradation as being the result of the additional mental workload required to remember the 
satisfactory conditions for switching response types, changing mental models — and in some cases inceptors — for 
their interaction with the vehicle, making the physical switchover of the response type with the RHC slider switch, 
and the transient behavior of the vehicle as the new control mode became active or as the pilot changed their focus 
on the displays. It is possible that additional training would remove some of the mental effort required to change 
response types, or that giving the pilot more flexibility in deciding when to switch response types would improve the 
handling qualities ratings in comparison with the pure response types, but at the level of training present for this 
experiment it was not useful to have a second response type available. 
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Figure 7. Cooper-Harper ratings for Velocity Increment mode with and without guidance 


Pilots generally prefer to be able to accomplish a task without relying solely on guidance, and previous studies 
showed that the precision landing task with RCAH requires either guidance or a large amount of training. The VINC 
response type, in contrast, was well suited to this task with or without guidance as shown in Fig. 7. That figure 
shows only the configurations in which VINC was the up-and-away response type, and groups the same vertical 
descent response types together with and without guidance. Differences in handling qualities ratings are minor 
between the guided and unguided cases and pilot comments confirmed that velocity guidance was only really useful 
as a training aid; once the pilot had flown about a dozen runs with VINC they tended to rely more on state data than 
guidance and used piloting techniques they developed on their own. 
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Figure 7 shows that the most useful of the vertical descent response types for VINC mode was IPC-TD, the 
Incremental Position Command with the touchdown designator. While pilots did not like being required to switch 
response types at fixed values of range and velocity, they did report a preference for having the option to change to 
IPC-TD once they had established the vehicle in a stable vertical descent. This was preferred because the IPC-TD 
and IPC systems tended to exhibit excessive overshoot when position commands exceeded 25 ft, and because once 
position hold was engaged there was little for the pilots to do. They reported that the only configurations in which 
they were able to look out the window and devote time to acquiring reasonable situational awareness about their 
trajectory were the configurations with IPC or IPC-TD. 

The pilot ratings and comments suggest that VINC mode should be optimized and examined in more detail as a 
way to improve handling qualities without increasing the size of the RCS thrusters. This is the only alternative 
method found to date that would provide overall Level 1 handling qualities (defined as greater than 60% of pilots 
giving a CHR of 1-3), and it is the only known method that can achieve this without the use of explicit guidance 
cues. The IPC-TD system should also be investigated as an optional feature for pilots to use when they feel 
comfortable turning position control over to the automatic system. More sophisticated control systems can reduce 
the overshoot for large position increments, and allowing pilots the freedom to change response types at any time 
would provide them the freedom to develop their preferred flying techniques. 

NASA Task Load Index Ratings 

The utility of the NASA Task Load Index (TLX) lies in the quantification of workload in six different categories, 
which allows the experimenter to determine what type of workload is contributing to the difficulty of the task and 
not simply how much total workload is involved. This distinguishes the TLX scale from the Cooper-Harper scale 
because performance and workload are not jointly considered in the selection of a single rating, but are explicitly 
rated independently. 



(a) (b) 

Figure 8. TLX ratings for selected configurations 

The TLX ratings for the pure up-and-away response types are shown in Fig. 8a, and the comparison of guided 
and un-guided pure VINC modes in Fig. 8b. In parallel with the CHR results (see Fig. 6 and Fig. 7), the VINC mode 
receives better ratings than the pure RCAH mode when guidance is present. Temporal demand and performance are 
rated about the same between the two configurations, but the other four elements of workload are all slightly better 
with the pure VINC mode. Also paralleling the CHR results are the similarities between the guided and un-guided 
pure VINC modes; without guidance the workload categories are rated slightly worse that the guidance cases, but 
the differences are very small. The main conclusions discussed in the Cooper-Harper section are confirmed by the 
TLX ratings: pure VINC mode is rated more favorably than pure RCAH mode, and guidance does not provide 
significant improvements in workload for the pure VINC mode. 

The TLX ratings provide new insights in comparison with CHRs when the use of a vertical descent response 
type is considered. Figure 9 shows the six components of TLX for the RCAH response type (Fig. 9a) and VINC 
response type (Fig. 9b) for each of the vertical descent types with guidance available. While the pure modes do well 
in comparison with the mixed modes, the conclusion from the CHRs that they are the best modes is not supported by 
the TLX data. In the cases of both RCAH and VINC up-and-away, the IPC-TD response type is the clear favorite 
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in every category except performance. The reasons for this were given in the Cooper-Harper Ratings section and 
include the principal fact that in most cases with this vertical descent response type the pilot was able to spend the 
last tens of seconds of the simulation run looking out the window and gaining situational awareness about the 
vehicle, trajectory and landing site. In no other response type was this possible. The lower performance ratings 
received by the IPC-TD response type is probably due to the already-mentioned tendency for the control system to 
overshoot in position when large position increments were commanded. These results indicate that position 
command response types should receive further examination. 



Figure 9. TLX ratings for guidance configurations 


Pilot- Vehicle Performance 

The handling qualities ratings indicate the overall level of compensation and performance a pilot experienced in 
the execution of a particular task. It is necessary to examine the objective performance as determined by the 
numerical performance parameters to help determine whether it was the compensation or the performance itself that 
was the principal factor in the handling qualities rating of a particular configuration. The most important 
performance parameter for this precision landing task, given that very few of the touchdowns would have resulted in 
the vehicle tipping over, was the range to the target at touchdown. The most important parameter from a vehicle 
design perspective is the amount of propellant required to execute the task. 
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The median, mean and absolute ranges of all the pilots’ radial offsets at touchdowns are shown in Fig. 10. Here, 
radial offset is defined as the horizontal distance between the centerline of the vehicle after it came to rest on the 
lunar surface and the center of the landing site; the desired range was 15 ft, adequate was 25 ft. The maximum range 
for many of the configurations is not shown because the pilot landed far from the touchdown site, well outside the 
adequate range and often because he had given up on achieving good touchdown performance for that particular run. 
The presence of these large range errors emphasizes the difficulty of the precision landing task and ever-present 
possibility of inadequate performance even after a string of successful runs. Figure 10 also demonstrates the utility 
of having a position command response type (IPC) and, more specifically, one that shows the actual designated 
touchdown point (IPC-TD): while the handling qualities ratings were generally best for pure response types, the use 
of IPC-TD improved the median touchdown accuracy over the pure types. Finally, the pilots commented that 
following explicit attitude guidance with RCAH required significant attention and so received relatively poorer 
handling qualities ratings than the VINC modes, but that additional attention appears to have resulted in more 
accurate touchdowns. 



The amount of RCS propellant consumed by the principal experiment configurations is shown in Fig. 1 1 . That 
figure shows there is little difference in RCS propellant consumption across all these experiment configurations, a 
conclusion that is reasonable because the amount of consumed propellant is dependent on the changes in the vehicle 
attitude during the reference trajectory, and each configuration resulted in approximately the same set of roll and 
pitch dispersions. This result is useful to a lunar lander designer because it suggests that the control response type 
may be selected without having to worry about its effect on propellant mass. The descent engine propellant 
consumption showed even less variability among the configurations. The only circumstance in which additional 
descent engine propellant was consumed was when a large position error during vertical descent drove the pilot to 
reduce his descent rate and therefore spend additional time and propellant executing the task. 

Conclusions 

Eleven NASA astronauts and test pilots participated in a simulation evaluation of the handling qualities of a 
lunar lander during the final phase of a precision approach and landing using a variety of control response types and 
guidance and display features. The control response types included the baseline Rate Command Attitude Hold type 
used by the Apollo LM, a more highly-augmented Velocity Increment response type that allowed the pilot to 
command velocity during the up-and-away, decelerating portion of the flight, and three response types designed for 
use only during the vertical descent portion of the trajectory in which the pilot commanded translational 
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acceleration, velocity or position. The displays included features adapted from aircraft glass cockpit and advanced 
rotorcraft designs, and configurations were presented to the pilot with attitude guidance, with velocity guidance, or 
without guidance of any kind. The principal goal of the experiment was to determine whether Level 1 handling 
qualities, corresponding to Cooper-Harper ratings between 1 and 3, could be achieved with low control power for 
some combination of these control response types and display features, or whether the known method of increasing 
the vehicle’s attitude control authority was the only way to achieve Level 1 handling qualities. Secondary goals 
were to find response types that could be flown without guidance and determine the touchdown range and propellant 
consumption requirements as a function of the experiment variables. 

The new Velocity Increment Command response type was given Cooper-Harper ratings between 1 and 3 by two- 
thirds of pilots, making it overall Level 1 and representing a substantial improvement over Rate Command with 
Attitude Hold. Whereas the current simulation and several previous experiments have consistently shown that this 
latter response type, used by the Apollo LM but with higher control authority, is given Level 2 ratings, only the 
Velocity Increment Command response type tested here has received a majority of Level 1 ratings. In addition to 
improved handling qualities with velocity guidance presented to the pilot, the Velocity Increment response type 
received almost identical ratings whether guidance was on or off; pilots reported that they used the guidance only to 
help develop a flying technique of their own and then tended to disregard the velocity guidance. In contrast, the 
baseline rate command system proved extremely difficult to fly without guidance given the limited amount of 
training time available to each participant. The only negative comments received about the Velocity Increment 
response type were objections to giving up direct control of the vehicle’s pitch and roll angles; two pilots mentioned 
this factor as a consideration in their handling qualities rating. 

The response types designed for use in the vertical descent portion of the trajectory did not improve handling 
qualities, with one possible exception, and generally showed satisfactory performance. While these response types 
were optimized for the small translation maneuvers required in the final descent, the additional mental effort 
required to change flying techniques, monitor different instruments or even change inceptors outweighed most of the 
benefits of these additional control modes. The Incremental Position Command response type (used in the vertical 
descent portion of the trajectory) received better workload ratings than the pure response types because the pilot had 
time to look out the window and enlarge his instrument scan during the last tens of seconds of the task, an 
opportunity that was not present with either Velocity Increment Command or Rate Command with Attitude Hold. 
This vertical descent response type also improved range to the target point at touchdown, and so might be a useful 
feature available to the pilots to use on an optional basis. The configurations studied in this experiment, and in 
particular those configurations that received the best handling qualities ratings, used roughly equivalent amounts of 
reaction control system propellant, suggesting that the control response may be selected without concern for 
impacting system mass requirements. 
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